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ABSTRACT 

During academic year 2001-02, approximately 165 students in 
eight sections of Introductory Spanish and French completed weekly online 
quizzes as part of their final grades. While most of them completed 
traditional quizzes, two third-semester Spanish classes completed multimedia 
listening comprehension quizzes based on audio and video from the Web. All of 
the quizzes were accessed from the University’s Blackboard account. Students 
were instructed to log in and follow the on-screen instructions to complete 
the assignment. This paper focuses on creation of the traditional quiz; 
creation of the multimedia quizzes; and saving and making available the 
quizzes. It also presents results from anonymous online student surveys that 
asked about students’ experiences with the digital quizzes. Results indicate 
that the weekly quizzes were viable for foreign language classes. Because the 
students successfully manipulated the new medium, between 10-15 minutes of 
class time were saved each week, which were more profitably used in the 
mastery of course material. Instructor time was also greatly conserved since 
the quizzes were self-correcting and self-tabulating. Disadvantages to using 
the Blackboard system included the integrity of the examination process 
itself (since students were unsupervised during their quizzes) and, for the 
quiz creator, locating stable, appropriate, and archived audio/video for the 
multimedia quizzes. (Contains 9 references.) (SM) 
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Using Blackboard 5 to Deliver Both Traditional and Multimedia Quizzes on-line 

for the Foreign Language Class 

During the Academic Year 2001-2002, approximately 165 students in eight sections 
of Introductory Spanish and French were asked to complete weekly on-line quizzes as 
part of their final grades (15% of the total). While the majority of these students 
completed traditional quizzes (comprised of multiple choice, fill-in-the blank, etc.), two 
third semester Spanish classes completed multimedia listening comprehension quizzes 
based on audio and video from the web. 

All of the quizzes were accessed from the University’s Blackboard account and 
students were instructed to log-in and follow the instructions on screen to complete the 
assignment. 

I. Getting started - The Creation of the Traditional Quiz 

The creation of on-line quizzes using Blackboard is a relatively easy to use process 
which presents only a few more steps than the actual typing of the quiz. Once the 
instructor has logged on to Blackboard using his/her User ID and password, the first step 
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Figure I: Instructor’s View of the Welcome Screen. 

is to locate the Control Panel at the lower left hand comer of the Welcome page as 
shown in Figure I. 

The Control Panel is a part of the Blackboard Welcome page that is only available to 
the instmctor. By accessing the Control Panel, the instmctor controls all aspects of the 
Blackboard course page. Once the Control Panel is opened, the Assessment Manager is 
seen at the right of the screen. Clicking on the Assessment Manager leads to the notation 
“Add Quiz/Exam.” You are now ready to write your on-line quiz after the completion of 
three preliminary steps: First, the quiz must be named (or numbered); second, a 
description of the assessment must be given. The description need not be long or 
complicated, in fact, a notation like “first conjugation verbs,” or “definite articles” will 
suffice. The third step, again not onerous, involves providing the students with 
instmctions for completing the quiz. The instmctions can be purposely vague, at this 
point, however, unless the instmctor has a precise idea of the type of questions which will 
be asked. Since my quizzes were often a mixture of 3 of the different question types, I 
delayed writing the instmctions until the end of the quiz writing process. . 

These preliminary steps, which have taken a few words to explain, are 
accomplished in a few seconds of pointing, clicking and submitting. It is now time to 
consider what type of question to write. As alluded to above, Blackboard provides 
significant latitude since the platform allows for 7 question types (six of which are self- 
correcting). I found three different types to be useful in a foreign language course. The 
Fill-in-the-Blank option turned out to be much more than the simple placing 
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of one correct word in a blank. Indeed, as shown in Figure II, multiple words could be 
placed in one blank, as long as each word was separated by a space. This has great utility 
for any translation exercise where word order is immutable. The Multiple Choice option 
allowed for the testing of all sorts of material and the instructor has the option of 
providing up to 20 possible choices. As shown in Figure III, the True False option could 
be easily adapted to test discernment of correct grammatical construction. 

Since it was important that the quizzes be self-correcting, I did not use the Short 
Answer/Short Essay format which must be graded by hand. This option, however, could 
be profitably used for compositions and other exercises requiring longer and more 
personalized answers. 

It is obvious that not all types of questions are equally well-suited to the material to 
be tested or to the degree of difficulty desired. Having said this, however, it should be 
noted that almost all the standard material taught in lower-level language classes is 
suitable for testing using Blackboard. Testing of verb conjugations can be easily done 
using the Multiple Choice option or by requiring the students to fill in the correctly 
conjugated verb with the Fill-in-the-Blank option. It sometimes happens that students or 
instructors are obliged to produce diacritical marks during the course of the quiz. This is 
far from an insurmountable difficulty for students using a desktop computer. By 
simultaneously depressing Alt and the ASCII code on the numeric keypad (130 gives e, 
for example), the desired character will appear. Things are a bit problematic for users of 
laptops, however. Prior to pressing Alt, they must first engage the system by pressing 
Shift and Number Lock and only then simultaneously depress Alt and the ASCII Code on 
the keys which display the superimposed numbers. (There is no numeric keypad on a 
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laptop.) Since not all students were able to follow these instructions, quizzes were 
generally written which did not require the use of diacritical marks. 



Name: Quiz #2 



Instructions: Translate the words you see into Spanish Pay attention to agreement as we discussed 

in class. Take the quiz after reviewing your notes from August 3D. Pay attention to the 
use of the definite and indefinite article. 



Question 1 


Fill in the Blank 


(10 points) 




Question: A red table 

i — . ; 




Question 2 


Fill in the Blank 


(10 points) 




Question: a purple backpack 

i .j 




Question 3 


Fill in the Blank 


(10 points) 



Figure II. Fill-in the-blank questions testing adjective agreement 

Adjective agreement, article agreement and the formation of singular and plural, 
concepts which consume so much of the beginning students ’energy, are easily reinforced 
using the Fill-in-the-blank option. Students must fill in specific words in a certain order 
to receive full credit. An example from an actual on-line quiz (Figure II) shows the ease 
with which the quiz can be written by choosing the “Fill-in-the blank” option from the 
scroll down menu and clicking to submit. 

Since I wanted first semester students to realize that adjectives generally follow 
nouns and must follow the rules of agreement, I asked that the students type in the answer 
in Spanish or French. Since the system will reject any words not in exact sequence, I 
found this to be a good test of the students’ mastery. The rules of capitalization need not 
be respected since this option is not case sensitive. This example illustrates both the ease 
of quiz making as well as one of the potential drawbacks: as constructed, the quiz allows 
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for no partial credit. While an instructor will immediately recognize that an individual 
student has mastered agreement but not adjective positioning or vice versa, the computer 
program will not and will not accord partial credit. Instructors wishing to modify scores 
can always do so by accessing the students’ entry in the on-line gradebook, located in the 
same Assessment section used to create quizzes.. 



Instructions: For questions 1-4, identify the correct sentence. If the first sentence is correct, click T, if 

the second sentence is correct, click U F U . For the remaining sentences, conjugate the 
French verb. 



Question 1 True /False (10 points) 

Question: 1. Sur le lundi, je dine au restaurant. 

2. Le lundi, je dine au restaurant. 

C True 

O False 

Question 2 True /False (10 points) 

Question: 1 . Ma famille habite a Bruxelles. 

2. Ma famille habitent a Bruxelles. 

0 True 

C False 

Figure III. True/False question format used as Grammaticality Test 

I found that the typical True/False question, never much in evidence in foreign 
language classes, offered a unique question format to test the students’ ability to discern 
grammatical differences. (See Figure III) Students were presented with two sentences in 
the foreign language that normally differed in only one detail. They were then asked to 
choose “T” if the first sentence was correct and “F” if the second sentence was correct. 

I felt that these grammaticality judgment tasks, borrowed from the field of SLA 
research (5) tested students’ reasoning about grammatical structures that had been 
presented in class and were less mechanical than those quizzes which only required them 
to fill in the blanks with conjugated verbs. 
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II. The creation of multimedia quizzes 

The creation of the weekly multimedia quiz greatly interested me. This was not 
only because it was possible to set up such a quiz but because of the larger implications 
that such a “delivery system” as Blackboard could have on the investigation of auditory 
processing. Since my focus was on listening comprehension in my third semester 
classes, I searched the web to find audio (and later video) that would be appropriate for 
classes at this level. (Except in one instance, I did not assign any multimedia quizzes to 
my first or second semester classes.) 

At the end of the first week of creating audio quizzes I learned an important lesson, 
which, however, greatly limited the web material I could use. While many countries 
throughout the world broadcast in French and Spanish, these broadcasts (mainly, news) 
are located in what is called a temporary file so that, while BBC Mundo (BBC’s Spanish 
service at http://www.bbc.co.uk/hi/spanish/news) broadcasts the news in Spanish on an 
hourly basis, the 1 1AM broadcast is placed in a temporary file that is replaced by a new 
broadcast at noon. This means that a listening comprehension quiz based on the 1 1AM 
broadcast will be totally obsolete if the students take it at noon. 

The construction of the multimedia quiz with audio and video material from the 
web is more labor intensive than the writing of the traditional quiz. For one thing, it 
requires not only that a quiz be written (in this instance of the listening comprehension 
type) but also that suitable material be identified and accessed from the web. The 
suitability of the web material depends not only on the level of the class but also on the 
availability and stability of the video or audio clip. In fact, stability of the link is actually 
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the more important factor because, while the instructor can easily control the level of 
difficulty of the quiz, he/she is powerless in the face of a link which is unstable. 

I soon learned that the type of stable audio or video file needed was called an 
“archived file.” Archived files are those that the server keeps on-line for a period of time 
(and does not take off-line after a few hours). Since the quizzes were done on a weekly 
basis, it was necessary to find web files that would stay on line for at least 6 days, thus 
allowing the students approximately a week to listen to and/or view the material before 
taking the quiz. 

As sometimes happens, a solution was found in an unexpected place. The White 
House in Washington, D.C. announced in May of 2001 that each of the President’s 
Saturday morning weekly radio addresses would now be broadcast in Spanish as well as 
English. An investigation of the White House site (www.whitehouse. gov) revealed that 
not only was each radio broadcast delivered by a native speaker— President Bush 
exceptionally delivered the first speech in Spanish on May 5, 2001— but more 
interestingly, from a pedagological standpoint, a complete Spanish transcript was 
provided for each radio address. Students could only benefit by listening to the audio and 
following along with the transcript. In addition, the fact that the weekly broadcasts were 
extremely topical would facilitate understanding. 

For a six or seven week period consecutively students were required to listen to a 
radio address that had been delivered the previous Saturday morning. Soon after each 
Saturday’s address, I would listen to the speech and write 10 comprehension questions 
based on the content of the speech. Sometimes these questions were True/False and other 
times they were multiple choice. Since the weekly addresses were available in English 
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Discurso Radial Del Presidente a la Naclon 

■(Escucha el Discurso Radial del Presidente con Real Audio Player 

Buenos dt'as. Quiero informarles sobre el progreso logrado en las muchas 
frentes de nuestra guerra contra el terrorismo. Esta es una guerra 
diferente, que libraremos agresiva y metddicamente para quebrantar y 
destruir la actividad terrorista. 

Recientemente, muchos miembros de nuestras fuerzas militares han dejado 
sus hogares y sus familias y han comenzado a prepararse para misiones 
futuras. Miles de reservistas han sido llamados a servicio activo. Soldados, 
marineros, aviadores, infantes de marina y guardacostas estan siendo 
enviado a lugares alrededor del mundo, listos a responder cuando su pals 



Figure IV. The President’s Weekly Radio Address in Spanish. 



on the same website, I wrote all the questions in Spanish. The goal was that the students 
be exposed to good, clear Spanish and that this task be reinforced by having the transcript 
in front of them. 

While I felt that the White House web site offered a good starting point for a semester 
of multimedia quizzes, I was not convinced that it represented a true test of listening 
comprehension. Students who wanted to could simply scan the transcript and answer the 
questions without ever having listened to any Spanish at all. In this case, the exercise 
would be a reading comprehension exercise only, which, although useful, did not 
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increase student exposure to spoken Spanish. The time had nevertheless come to find a 
new site that would more closely be an exercise in listening comprehension. 

Occasionally, there is an additional obstacle to be dealt with in presenting students 
with multimedia quizzes: Students have difficulty downloading the RealPlayer plug-in 
without which they will be unable to hear or see topics presented. A greater problem and 
the single most recurrent one is that students become “locked out” of the Blackboard 
quiz site after listening to or viewing the assigned website from the External Links 
section — the default setting allows for only one quiz entry. The only remedies are for the 
instructor to reset the quiz on the individual grade report allowing the student to take the 
quiz again or instruct students to minimize the screens. During the second semester, 
therefore, I decided to allow the multimedia quiz takers to have multiple entries to each 
week’s Blackboard quiz . This option, provided for in the “Save and Make Available” 
menu, totally solved the problem but at a cost which will be discussed later. 

The Voice of America, a US governmental agency, ('www.voa.gov .'). 
broadcasts news and cultural programming from its studios in Washington, D.C. 
Considering the breadth of its foreign language broadcasting, it is a clear rival to the BBC 
World’s Service. VOA broadcasts to Latin America include a weekly television program 
called “Conversemos” which is broadcast live by satellite and archived on the internet for 
one week. 

The “Conversemos” program turned out to be an excellent vehicle for a multimedia 
listening quiz using the Blackboard platform. Since the program is archived for 7 days 
(from Thursday evening to the following Thursday evening) it allowed the students 
ample opportunity to view the broadcast and answer the comprehension questions which 
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had been written in English. Precisely because the comprehension questions were written 
in English, students could focus on listening to the content of the program without being 
impeded by their Spanish language reading levels. (As had sometimes been the case for 
The White House quizzes.) 

The broadcasts were made available on Blackboard by providing a link in the 
External Links section (see Figure I). Students were instructed to view 10 minutes of the 
program (by consulting the clock on the RealPlayer screen) and then answer the self- 
correcting listening comprehension questions. The VOA link, unlike many other 
broadcast sites such as those of Chilean, Venezuelan and Argentine TV, was highly and 
consistently stable. For the test creator, there was an additional benefit to the VOA link: 
It needed to be entered only once in the External Links section since VOA “rotated” the 
“Conversemos” program out of the same link each week. 

There are several other advantages to the VOA program: one is that the 
programming is topical. In the weeks following the attacks of September 1 1 th , 
“Conversemos” focused on bioterrorism -a subject many in the US were extremely 
familiar with. At times, however, the programs were far removed from my students’ 
knowledge base — a program focusing on the FARC of Colombia could not be profitably 
used. 

Over all, however, the advantages outweighed the disadvantages. Of special interest 
is that the web broadcasts present both native and highly proficient non-native speakers. 
The producers of “Conversemos” frequently invite US experts with near-native 
proficiency to participate in its round table discussion. Since “Conversemos” is also a 
call-in program, students are exposed to other accents from Central and South America. 
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In one instance, the students were quick to note that a caller from Peru spoke differently 
from the other panelists of that week’s program. 

The use of “Conversemos” was an unqualified success as a listening 
comprehension exercise and it met all the criteria that I had set: the video and audio were 
clear, the link was stable, and the subject matter was topical and interesting. 

III. The Final Step — Saving and Making Available 

Before students can take the quiz that has been written, it must be “saved and 
made available,” in other words, “put on line.” This process is begun by clicking on the 
aptly named “Save and Make Available” radio button which always appears at the end of 
each quiz. If you wish only to save the quiz but are not yet ready to put it on-line, click on 
the “save” radio button only. 

Care must be taken here both with the body of the text and the quiz settings. Proof 
your quiz carefully: Over the course of writing approximately 50 quizzes during an 
academic year, some serious errors were made which included the inadvertent providing 
of wrong answers and the misassigning of point values for the quizzes. Perhaps the 
greatest drawback to the Blackboard system is that these and other errors cannot be 
corrected after the fact. Unlike a hard copy quiz which can be easily amended in class, 
this electronic sort is unalterable once on-line. Two remedies are available, however. 

One, is to make the quiz unavailable and lose all grades thus far posted or posting an 
announcement in the quiz directions alerting the students to the error which has been 
made. 
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Once you have thoroughly checked your quiz and clicked on the “Save and Make 
Available” radio button, a new page will appear asking if you want to make the 
assessment available. This will generate an announcement informing the students of the 
presence of a new quiz. Since the announcement appears on the course’s Welcome Page 
and invites them to take the quiz, I have always found it to be useful. 

One of the great benefits of the Blackboard system is that the quiz can be put on a 
“timer” and programmed to go on and off-line. It will always be available to the 
instructor, however. Since the students were on a strict weekly schedule of quizzes, the 
quizzes were programmed to be activated at noon Friday and be deactivated at noon the 
following Friday. Other worthwhile options to be checked on the Save and Make 
Available Page include “Show Detailed Results” and “Reveal Correct Answer ’’which 
allow students to see the correct answer after finishing the quiz — this is a great learning 
tool. The button “Feedback Enabled” allows the instructor to provide explanatory 
comments for the incorrectness of an answer given or praise for a correct one. In an 
exercise of the French partitive, a student who had filled in “de 1’ ” in the sentence “Je ne 

bois pas eau minerale,” would have not only received the correct 

answer of “d’ ” but would have been reminded that the sentence is negative and precedes 
a vowel and thus requiring ”d’.” This raises the level of the quiz to a tutorial rather than a 
simple assessment. 

On the other hand, the option to “allow multiple attempts” is a problematic on; 
while useful in exercises where students are liable to be “locked out” (a particular risk 
during the multimedia listening exercises), it allows all students to retake the quiz any 
number of times and get a perfect score. Perhaps future incarnations of Blackboard will 




14 



13 



allow the instructor to specify the exact number of attempts allowed as well as provide 
score averaging. While it is not a bad thing in itself to award student diligence, allowing 
unlimited attempts does little to help the instructor evaluate the students’ progress over 
time, nor does it encourage students to improve their performance on their own. The last 
option on the Save and Make Available page allows the instructor to set a time limit for 
the quiz. While I set these controls during the first semester (for 15 to 20 minutes), 
students expressed anxiety about seeing a ticking clock on the screen. Since student 
surveys at the end of the first semester revealed that students had spent no more than 15 
minutes anyway doing traditional quizzes anyway I do not feel it appropriate to limit the 
time allotted for each assessment. This may not, of course, be the case with your group of 
students. 

IV. The results of student surveys 

At the end of each semester, I asked the students to complete an anonymous on-line 
survey on Blackboard regarding their experiences with the digital quizzes. In order to 
induce their participation during the busy days following the last day of classes, I granted 
each respondent a 1% point addition to their final grade. I stressed that their responses 
would be completely anonymous and that Blackboard would reveal their participation 
without revealing their answers. 

Since there were two types of assessments given, traditional and multimedia, there 
were two types of surveys. Additionally, from my conversations with students, I knew 
that many had never taken a Blackboard quiz or if they had, that it had been on a very 
sporadic basis. The first questions on all the surveys, therefore, for the purposes of setting 
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a baseline was to determine the percentage of students who had never taken a Blackboard 



quiz. 





Fall 2001 


Spring 2002 


Question: 


Sp. 1 


Sp. 3 


Fr. 1 


Sp. 2 


Sp. 3 


Fr. 2 


Prior to this course, or 
another course with this 
same instructor, I had 
never taken a 
Blackboard quiz. 


79% 


69% 


86% 


64% 


94% 


92% 




N=38 


N=16 


N=14 


N=39 


N=16 


N=12 



Table I. Percentage of Students who had never taken a Blackboard Quiz. 



The numbers in Table I clearly reflect a great untapped opportunity to use 
Blackboard for assessment purposes. When reviewing these figures, it is significant to 
note that the majority of the surveyed students were not freshmen but were 
upperclassmen who had delayed fulfilling their language requirement until their junior or 
senior years. Despite their seniority on campus, then, few had had any experience with 
Blackboard quizzes prior to enrolling in the Spanish or French classes in question. 

The students were also surveyed on the initial level of difficulty they encountered 
using the Blackboard platform as shown in Table II. The multiple choice responses to the 
question: “Initially, did you find the Blackboard system, very easy, moderately easy, 
etc?” is found in Table n. 
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Question: 


Fall 2001 


Spring 2002 


Initially, did you find the 
Blackboard System? 


Sp. 1 


Sp.3 


Fr.l 


Sp.2 


Sp.3 


Fr.l 


Very easy to use 


42% 


31% 


29% 


49% 


50% 


33% 


Moderately easy to use 


50% 


38% 


57% 


46% 


44% 


58% 


Difficult to use 


8% 


19% 


14% 


5% 


6% 


8% 


Very difficult to use 


0% 


12% 


0% 


0% 


0% 


0% 


Total percentage finding 
Blackboard difficult or very 
difficult to use: 


8% 


31% 


14% 


5% 


6% 


8% 




N=38 


N=16 


N=14 


N=39 


N=16 


N=12 



Table II. Initial Perception of degree o: 
sections. 



f difficulty of Blackboard quizzes — all 



If one looks only at the percentages responding that Blackboard quizzes 
were “difficult” or “very difficult,” the negatives are quite low with the data for 2002 
being even lower than the previous semester. This decrease in the level of difficulty 
experienced is due partly to the fact that at least a third of the students in French 2 and in 
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Spanish 2 were students who had continued with this same instructor from the Fall 
Semester 2001 and thus were “accustomed” to the Blackboard system. Another factor, 
intangible though it may be, was that the instructor herself was much more at ease and 
confident with the system and that this ease translated into the students’ expectation that 
the Blackboard assessments were not difficult. 

One cannot help but notice, however, that nearly a third (31%) of those students who 
had taken the multimedia quizzes (Spanish 3, Fall semester 2001) found such a system 
difficult to very difficult to use. This relatively high percentage was due to the fact that 
many students could not successfully navigate between the audio/video content which 
was housed in the External Links section of the Welcome Page (see Figure 1) and the 
comprehension quizzes. If a student disregarded directions and went first to the quiz 
(which might appear intuitive) before doing the viewing/listening, the student could find 
himself “locked out” and unable to enter the assessment for the second time. While many 
students were able to avoid this pitfall by minimizing each screen, it was decided in the 
Spring semester that all students would be allowed multiple entries to the quiz site. (This 
is easily done by checking the multiple entries option on the Save and Make Available 
Page.) This was done to ensure that the students would concentrate on the video/audio 
content without fear of the mechanics of the quiz procedure which strictly disallowed 
more than one attempt at a quiz. 

Allowing multiple entries is ultimately not a viable alternative, however, since the 
current Blackboard system does not average grades but merely allows students to retake 
the quiz as many times as needed to attain a perfect score. Multiple entries, in fact, mean 
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unlimited entries. A perfect score could be achieved by taking note of the incorrect 
answers and resubmitting the quiz with the correct answers. 

On a more encouraging note, students responded extremely positively to the overall 
usefulness of the on-line quizzes as shown in Table III. 





Fall 2001 


Spring 2002 


Question: 

Which of the following 
statements most accurately 
describes your overall 
experience with the 
Blackboard quizzes? 


Sp. 1 


Fr. 1 


Sp. 2 


Fr.2. 


A) Blackboard quizzes 
were a complete waste of 
time. 


3% 


7% 


0% 


8% 


B) The quizzes helped a 
little bit to review. 


24% 


21% 


18% 


25% 


C) The quizzes helped to a 
moderate degree to review. 


39% 


36% 


36% 


42% 


D) The quizzes helped a lot 
in the review process. 


34% 


36% 


46% 


25% 


Overall positives=Total of 
B, C and D. 


97% 


93% 


100% 


92% 




N=38 


N=14 


N=39 


N=12 



Since the focus of the multimedia quizzes was on listening comprehension, the students 
who took these assessments were asked to rate the overall usefulness of the quizzes in 
improving listening skills: 
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Question: 

What was your overall 
experience with Blackboard? 


Sp. 3 
Fall 2001 


Sp.3 

Spring 2002 


It was a complete waste of 
time. 


25% 


6% 


It helped a little bit to listen to 
real Spanish 


25 % 


50% 


It helped a lot to listen to real 
Spanish. 


44% 


19% 


It helped a lot to listen to real 
Spanish and it was interesting. 


6% 


25% 




N=16 


N=16 


Table IV: Perception of Overall 


Usefulness of Multimedia Quizzes. 



Conclusion 

Based on the results of the student surveys, the viability of the weekly 
Blackboard quizzes for foreign language classes seems assured. One of the original 
considerations of this study, indeed, the paramount one, was to save class time by 
requiring students to take electronic rather than weekly pencil and paper assessments. 
Because the students successfully manipulated the new medium, between 10 and 15 
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minutes of class time were saved each week which was more profitably used in the 
mastery of the course material. 

Not insignificantly, instructor time was also greatly conserved since the quizzes are 
self-correcting and self-tabulating with each student’s results being entered in the on-line 
gradebook. 

Additionally, the criterion of “value added” played a decisive role in the 
implementation and continuation of the pilot study. The application had to “add value” to 
the class experience in a way heretofore impossible. “Value added” was most clearly 
evident in the multimedia quizzes which allowed for new and easy ways to deliver 
audio/video from the web and simultaneously provide for self-correcting and self- 
tabulating listening comprehension quizzes. 

An additional “value added” feature which was not much utilized during the course 
of the Pilot Study was the feedback option which is accessible from the Save and Make 
Available Page— the last page to be completed before putting the quiz on line. The 
feedback option enables the instructor to add comments and/or explanations in the case of 
either a correct or incorrect quiz answers. Since these sorts of comments can only be 
entered manually (and laboriously) on each student’s hard copy, the platform provides a 
useful time-saving option which can only aid in student mastery of the subject matter. 

There are nonetheless disadvantages to using the Blackboard Assessment 
Program. For many, the greatest concern was the integrity of the examination process 
itself. Since students are unsupervised during the quiz there is no way to limit the amount 
or nature of the help received. Short of requiring that the students take the quizzes in a 
supervised locale, quizzes cannot be anything but open book. While a major obstacle to 
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some, this posed no real disadvantage to this writer because quizzes— even pen and paper 
ones— had always been viewed as learning exercises rather than as strictly graded events. 
Nonetheless, beginning students in the first and second semesters experienced traditional 
examination anxiety due primarily to the new format. This was quickly allayed 
midsemester when they were informed that the quizzes were going to be graded 
according a Pass/ Fail scale (70% and above). This helped focus the students’ attention 
on quizzes as review exercises for the in-class examinations. 

For the quiz creator, the greatest challenge was in locating stable, appropriate and 
archived audio/video for the multimedia quizzes. While Chilean, Argentine, French and 
other television networks do broadcast video using the RealPlayer plug-in (the only plug- 
in used in this project), these programs are either live or briefly archived which makes 
them impractical for class assignments. Other sites were found to be inappropriate for 
other reasons as was the case for the SCOLA site,( http://www.scola.org/ ) which 
provides an InstaClass lesson complete with reliable, archived audio from French and 
Mexican television complete with a transcript in the target languages. Because English 
translations are also provided, however, this SCOLA site could not be profitably used for 
listening comprehension exercises. 

Sometimes appropriate sites can be found by perusing the “archive” section of a 
web site. This was indeed the case of BBC Mundo which provided a good bit of archived 
audio by clicking on the Especiales link on the main page. This audio material was 
frequently available for periods longer than a month. Let it be clearly understood that the 
use of reliable links in the External Links section of the Blackboard platform poses no 
technological or bandwidth problems. What is consistently available to the instructor at 
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home on a dial up connection will be equally available to students who access the 
External Links section of the Blackboard site. 
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